Skip to content

Improved support for OpenRouter - #22

Merged
16francej merged 2 commits into
mainfrom
openrouter-base-model
Jul 30, 2026
Merged

Improved support for OpenRouter#22
16francej merged 2 commits into
mainfrom
openrouter-base-model

Conversation

@16francej

@16francej 16francej commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Stacked on model-provider-key-in-deploy.

A deployment that named OpenRouter (or OpenAI) as its model provider passed every deployment gate and then refused every message. The provider a deployment declares now selects the model it runs.

Change explainer: https://claude.ai/code/artifact/553b7a30-63bd-4f4b-931a-c75f6d2424c0

Root cause

defaultModelForHarness fell back to a provider-blind constant — claude-opus-5 — whenever no model was pinned. Provider availability was computed separately and used only to reject a model, never to choose one. So an OpenRouter-only deployment got an Anthropic default and every turn died on "that model isn't available on this deployment". check, plan, doctor and up were all green, because the OpenRouter key itself was fine.

The change

  • The base-model fallback is provider-aware, at the one function all eight resolution paths flow through. MODEL_PROVIDER reaches core as real configuration rather than only a secret gate.
  • Only the declaration moves a deployment. Without modelProvider, the shipped default stands exactly as before, so upgrading never changes an existing deployment's model or its billing.
  • Harness/provider pairs that can never serve a turn are refused at config load, in both the CLI and core, instead of at the first message. The CLI validates the provider core will actually use, so an env.core.MODEL_PROVIDER override is checked rather than skipped.
  • No model id has to be chosen at deploy time: qm init --model-provider openrouter scaffolds a config that works unedited.

Incidental

Two invariants in the base branch were held together by comments, against the repo's zero-comment standard; both are gone because the thing they explained is gone. Duplicate-name secret specs collapse to one per name (an any condition plus an optionalOtherwise flag), so array position no longer decides a secret's description. requiredWhen takes a list, retiring a dedupe pass and an if-chain fallthrough that treated any unmatched gate as the Linear one. Both catalogs gained uniqueness tests.

Verification

Two cores booted from this branch and its base, both HARNESS=pi MODEL_PROVIDER=openrouter with an OpenRouter key as the only credential:

base this branch
runtime-config.effective.modelId claude-opus-5 openrouter/auto
POST /v1/turns refused: its provider isn't configured reaches the OpenRouter API

The turn now fails only on the deliberately fake key (401 Missing Authentication header) — no real OpenRouter credential was available, so the provider round-trip is the one step not exercised end to end. qm init --model-provider openrouter and qm check were driven through the real CLI, including the new rejection of openrouter + HARNESS=codex.

CI was green on the first commit (14/14). CLI suite 484/484 locally; tsc clean on core, CLI and contract projects; eslint, oxlint --deny-warnings, prettier --check, knip clean. The root suite has a handful of local failures that are pre-existing timing flakes — the failing subset differs run to run, and the base branch fails the same files with a different subset.

Review

Three independent reviewers plus an adversarial Codex pass ran against the diff before it landed. Seven confirmed problems, all resolved. Four things were cut rather than repaired — each fix made the change smaller:

  • Durable base-model adoption on the first Admin provider key. Deleting that key later left the org pinned to a model it could no longer bill — worse than the previous fallback — and the write raced across instances and bumped the org revision.
  • Inferring the provider from whichever keys were present. It silently moved existing deployments: a stack holding an OpenRouter key as the documented optional fallback while running Anthropic from Admin would have flipped vendors, and bills, on upgrade.
  • Refusing a cross-vendor model pin at boot. The CLI holds no model registry and could not mirror the check, so this could brick a rollout qm check had already approved.
  • Filtering the surface-config model picker. It inverted an admin's explicit narrowing into a widening, because an emptied list falls through to every model. The picker the web UI actually reads already filters correctly.

Also fixed: a dropped de-duplication that listed two secrets twice in .env.example, and an env.core.MODEL_PROVIDER override that skipped CLI validation.

Known gap, deliberately not fixed here: a deployment that defers its key to the Admin page still needs a base model picked there by hand, and a durable selection outranks the declared provider. Both predate this change; deployment.md now says so instead of implying otherwise. The right fix is in runtime resolution — treat a stored selection whose provider is unavailable as absent — which also covers a key removed at redeploy.

🤖 Generated with Claude Code

Base automatically changed from model-provider-key-in-deploy to main July 30, 2026 06:08
16francej and others added 2 commits July 29, 2026 23:09
defaultModelForHarness fell back to a provider-blind constant whenever no
model was pinned. Provider availability was only ever used to reject a model,
never to choose one, so a deployment holding only an OpenRouter or OpenAI key
got an Anthropic default and refused every turn while check, plan, doctor and
up all passed.

The fallback is now provider-aware at the one function all eight resolution
paths flow through. MODEL_PROVIDER reaches core as real configuration rather
than only a secret gate, and availability is intersected with what the harness
can route. With no declaration the keys present decide, which fixes the same
defect for deployments that predate modelProvider. Harness and provider pairs
that can never serve a turn, and a model pin from another vendor, are refused
at config load in both the CLI and core instead of at the first message.

Two invariants the base branch held together with comments are gone with the
thing they explained: duplicate-name secret specs collapse to one per name via
an any condition and an optionalOtherwise flag, so array position no longer
decides a description, and requiredWhen takes a list, retiring a dedupe pass
and an if-chain fallthrough that treated any unmatched gate as the Linear one.
Both catalogs now have uniqueness tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codex challenged the diff and found two ways it could hurt a live stack.

Inferring the base-model provider from whichever API keys happen to be in
the environment silently moved an existing deployment, and its billing: a
stack holding OPENROUTER_API_KEY as the documented optional fallback while
actually running Anthropic from the Admin page would flip to openrouter/auto
on upgrade. The inference is gone. Without MODEL_PROVIDER the shipped default
stands exactly as before, so no deployment moves unless it says to.

Refusing a cross-vendor model pin at boot could brick a rollout the CLI had
already approved, because the CLI holds no model registry and could not
mirror the check. That throw is gone too; a pin wins, as it always has. What
remains is the harness/provider rule, which the CLI does mirror and a drift
test pins to the registry. The CLI now reads the provider core will actually
use, so an env.core.MODEL_PROVIDER override is validated rather than skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@16francej
16francej force-pushed the openrouter-base-model branch from 254d1ad to 37e28a1 Compare July 30, 2026 06:14
@16francej
16francej merged commit 9bf77f7 into main Jul 30, 2026
15 checks passed
@16francej
16francej deleted the openrouter-base-model branch July 30, 2026 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant